Creating and Manipulating Tensors

Learning Objectives:

Initialize and assing TensorFlow Variables
Create and manipulate tensors
Refresh your memory about addition and multiplication in linear algebra (consult an introduction to matrix addition and multiplication if these topics are new to you)
Familiarize yourself with basic TensorFlow math and array operations



In [1]:

    
import tensorflow as tf

Vector Addition

You can perform many typical mathematical operations on tensors (TF API). The following code creates and manipulates two vectors (1-D tensors), each having exactly six elements:



In [2]:

    
with tf.Graph().as_default():
    # Create a six-element vector (1-D tensor).
    primes = tf.constant([2, 3, 5, 7, 11, 13], dtype=tf.int32)
    
    # Create another six-element vector. Each element in the vector will be
    # initialized to 1. The first argument is the shape of the tensor (more on shapes below).
    ones = tf.ones([6], dtype=tf.int32)
    
    # Add the two vectors. The resulting tensor is a six-element vector.
    primes_plus_one = tf.add(primes, ones)
    
    # Create a session to run the default graph.
    with tf.Session() as sess:
        print(primes_plus_one.eval())









    



[ 3  4  6  8 12 14]

Tensor Shapes

Shapes are used to characterize the size and number of dimensions of a tensor. The shape of a tensor is expressed as list, with the ith element representing the size along dimension i. The length of the list then indicates the rank of the tensor (i.e., the number of dimensions).

For more information, see the TensorFlow documentation.

A few basic examples:



In [3]:

    
with tf.Graph().as_default():
    # A scalar (0-D tensor).
    scalar = tf.zeros([])
    
    # A vector with 3 elements.
    vector = tf.zeros([3])
    
    # A matrix with 2 rows and 3 columns
    matrix = tf.zeros([2, 3])
    
    with tf.Session() as sess:
        print("Scalar shape: {0} and the value:\n {1}".format(scalar.get_shape(), scalar.eval()))
        print("Vector shape: {0} and the value:\n {1}".format(vector.get_shape(), vector.eval()))
        print("Matrix shape: {0} and the value:\n {1}".format(matrix.get_shape(), matrix.eval()))









    



Scalar shape: () and the value:
 0.0
Vector shape: (3,) and the value:
 [0. 0. 0.]
Matrix shape: (2, 3) and the value:
 [[0. 0. 0.]
 [0. 0. 0.]]

Broadcasting

In mathematics, you can only perform element-wise operations (e.g. add and equals) on tensors of the same shape. In TensorFlow, however, you may perform operations on tensors that would traditionally have been incompatible. TensorFlow supports broadcasting (a concept borrowed from numpy), where the smaller array in an element-wise operation is enlarged to have the same shape as the larger array. For example, via broadcasting:

If an operand requires a size [6] tensor, a size [1] or a size [] tensor can serve as an operand.
If an operation requires a size [4, 6] tensor, any of the following sizes can serve as an operand:
- [1, 6]
- [6]
- []
If an operation requires a size [3, 5, 6] tensor, any of the following sizes can serve as an operand:
- [1, 5, 6]
- [3, 1, 6]
- [3, 5, 1]
- [1, 1, 1]
- [5, 6]
- [1, 6]
- [6]
- [1]
- []

NOTE: When a tensor is broadcast, its entries are conceptually copied. (They are not actually copied for performance reasons. Broadcasting was invented as a performance optimization.)

The full broadcasting ruleset is well described in the easy-to-read numpy broadcasting documentation.

The following code performs the same tensor addition as before, but using broadcasting:



In [4]:

    
with tf.Graph().as_default():
    # Create a six-element vector (1-D tensor).
    primes = tf.constant([2, 3, 5, 7, 11, 13], dtype=tf.int32)

    # Create a constant scalar with value 1.
    ones = tf.constant(1, dtype=tf.int32)

    # Add the two tensors. The resulting tensor is a six-element vector.
    primes_plus_one = tf.add(primes, ones)
    
    with tf.Session() as sess:
        print(primes_plus_one.eval())









    



[ 3  4  6  8 12 14]

Matrix Multiplication

In linear algebra, when multiplying two matrices, the number of columns of the first matrix must equal the number of rows in the second matrix.

It is valid to multiply a 3x4 matrix by a 4x2 matrix. This will result in a 3x2 matrix.
It is invalid to multiply a 4x2 matrix by a 3x4 matrix.



In [5]:

    
with tf.Graph().as_default():
    # Create a matrix (2-d tensor) with 3 rows and 4 columns.
    # | 5, 2, 4, 3 |
    # | 5, 1, 6, -2 |
    # | -1, 3, -1, -2 |
    x = tf.constant([[5, 2, 4, 3], [5, 1, 6, -2], [-1, 3, -1, -2]], dtype=tf.int32)
    
    # Create a matrix with 4 rows and 2 columns.
    # | -2, 3 |
    # | 2, 6 |
    # | -1, 1 |
    # | 0, 8 |
    y = tf.constant([[-2, 3], [2, 6], [-1, 1], [0, 8]], dtype=tf.int32)

    # Multiply `x` by `y`. The resulting matrix will have 3 rows and 2 columns.
    matrix_mul_result = tf.matmul(x, y)
    
    with tf.Session() as sess:
        print(matrix_mul_result.eval())









    



[[-10  55]
 [-14  11]
 [  9  -2]]

Tensor Reshaping

With tensor addition and matrix multiplication each imposing constraints on operands, TensorFlow programmers must frequently reshape tensors.

You can use the tf.reshape method to reshape a tensor. For example, you can reshape a 8x2 tensor into a 2x8 tensor or a 4x4 tensor:



In [6]:

    
with tf.Graph().as_default():
    # Create an 8x2 matrix (2-D tensor).
    x = tf.constant([[1,2], [3,4], [5,6], [7,8], [9,10], [11,12], [13, 14], [15,16]], dtype=tf.int32)    
    # Reshape the 8x2 matrix to 2x8 matrix.
    x_reshaped_2x8 = tf.reshape(x, [2, 8])
    
    # Reshape the 8x2 matrix to 4x4 matrix.
    x_reshaped_4x4 = tf.reshape(x, [4, 4])
    
    with tf.Session() as sess:
        print("Original matrix (8x2):\n{0}".format(x.eval()))
        print("Reshaped matrix (2x8):\n{0}".format(x_reshaped_2x8.eval()))
        print("Reshaped matrix (4x4):\n{0}".format(x_reshaped_4x4.eval()))









    



Original matrix (8x2):
[[ 1  2]
 [ 3  4]
 [ 5  6]
 [ 7  8]
 [ 9 10]
 [11 12]
 [13 14]
 [15 16]]
Reshaped matrix (2x8):
[[ 1  2  3  4  5  6  7  8]
 [ 9 10 11 12 13 14 15 16]]
Reshaped matrix (4x4):
[[ 1  2  3  4]
 [ 5  6  7  8]
 [ 9 10 11 12]
 [13 14 15 16]]

You can also use tf.reshape to change the number of dimensions (the "rank") of the tensor. For example, you could reshape that 8x2 tensor into a 3-D 2x2x4 tensor or a 1-D 16-element tensor.



In [7]:

    
with tf.Graph().as_default():
    # Create an 8x2 matrix (2-D tensor).
    x = tf.constant([[1,2], [3,4], [5,6], [7,8], [9,10], [11,12], [13, 14], [15,16]], dtype=tf.int32)
    
    # Reshape the 8x2 matrix to 2x2x4 (3-D tensor).
    x_reshaped_2x2x4 = tf.reshape(x, [2, 2, 4])
    
    # Reshape the 8x2 matrix to 16-element (1-D tensor).
    x_reshaped_16 = tf.reshape(x, [16])
    
    with tf.Session() as sess:
        print("Original matrix (8x2):\n{0}".format(x.eval()))
        print("Reshaped 3-D tensor (2x2x4):\n{0}".format(x_reshaped_2x2x4.eval()))
        print("Reshaped 1-D vector (16):\n{0}".format(x_reshaped_16.eval()))









    



Original matrix (8x2):
[[ 1  2]
 [ 3  4]
 [ 5  6]
 [ 7  8]
 [ 9 10]
 [11 12]
 [13 14]
 [15 16]]
Reshaped 3-D tensor (2x2x4):
[[[ 1  2  3  4]
  [ 5  6  7  8]]

 [[ 9 10 11 12]
  [13 14 15 16]]]
Reshaped 1-D vector (16):
[ 1  2  3  4  5  6  7  8  9 10 11 12 13 14 15 16]

Exercise #1: Reshape two tensors in order to multiply them.

The following two vectors are incompatible for matrix multiplication:

a = tf.constant([5, 3, 2, 7, 1, 4])
b = tf.constant([4, 6, 3])

Reshape these vectors into compatible operands for matrix multiplication. Then, invoke a matrix multiplication operation on the reshaped tensors.



In [8]:

    
with tf.Graph().as_default():
    # initialize given matrices
    a = tf.constant([5, 3, 2, 7, 1, 4])
    b = tf.constant([4, 6, 3])
    
    # reshape a to 2x3 and b to 3x1
    reshaped_a = tf.reshape(a, [2, 3])
    reshaped_b = tf.reshape(b, [3, 1])
    
    # result of multiplication of 2x3 and 3x1 matrices will be a 2x1 matrix
    mul_a_b = tf.matmul(reshaped_a, reshaped_b)
    
    with tf.Session() as sess:
        print("Multiplication of a and b:\n{0}".format(mul_a_b.eval()))









    



Multiplication of a and b:
[[44]
 [46]]

Variables, Initialization and Assignment

So far, all the operations we performed were on static values (tf.constant); calling eval() always returned the same result. TensorFlow allows you to define Variable objects, whose values can be changed.

When creating a variable, you can set an initial value explicitly, or you can use an initializer (like a distribution):



In [9]:

    
g = tf.Graph()
with g.as_default():
    # Create a variable with the initial value of 3
    v = tf.Variable([3])
    
    # Create a variable of shape [1], with a random initial value,
    # sampled from a normal distribution with mean 1 and standard deviation 0.35.
    w = tf.Variable(tf.random_normal([1], mean=1, stddev=0.35))

One peculiarity of TensorFlow is that variable initialization is not automatic. For example, the following block will cause an error:



In [10]:

    
with g.as_default():
    with tf.Session() as sess:
        try:
            v.eval()
        except tf.errors.FailedPreconditionError as e:
            print("Caught expected error: {0}".format(e))









    



Caught expected error: Attempting to use uninitialized value Variable
	 [[Node: _retval_Variable_0_0 = _Retval[T=DT_INT32, index=0, _device="/job:localhost/replica:0/task:0/device:CPU:0"](Variable)]]

The easiest way to initialize a variable is to call global_variables_initializer. Note the use of Session.run(), which is roughly equivalent to eval().



In [11]:

    
with g.as_default():
    with tf.Session() as sess:
        initialization = tf.global_variables_initializer()
        sess.run(initialization)
        # Now, variables can be accessed normally, and have values assigned to them.
        print(v.eval())
        print(w.eval())









    



[3]
[1.1409308]

Once initialized, variables will maintain their value within the same session (however, when starting a new session, you will need to re-initialize them):



In [12]:

    
with g.as_default():
    with tf.Session() as sess:
        sess.run(tf.global_variables_initializer())
        # These three prints will print the same value.
        print(w.eval())
        print(w.eval())
        print(w.eval())









    



[0.3052618]
[0.3052618]
[0.3052618]

To change the value of a variable, use the assign op. Note that simply creating the assign op will not have any effect. As with initialization, you have to run the assignment op to update the variable value:



In [13]:

    
with g.as_default():
    with tf.Session() as sess:
        sess.run(tf.global_variables_initializer())
        
        # Print the current value of v
        print(v.eval())
        
        assignment = tf.assign(v, [7])
        # The variable has not changed yet!
        print(v.eval())
        
        # Execute the assignment op.
        sess.run(assignment)
        # Now the variable is updated.
        print(v.eval())









    



[3]
[3]
[7]

Exercise #2: Simulate 10 rolls of two dice.

Create a dice simulation, which generates a 10x3 2-D tensor in which:

Columns 1 and 2 each hold one throw of one die.
Column 3 holds the sum of Columns 1 and 2 on the same row.

For example, the first row might have the following values:

Column 1 holds 4
Column 2 holds 3
Column 3 holds 7

You'll need to explore the TensorFlow documentation to solve this task.



In [15]:

    
with tf.Graph().as_default():
    # Creating a 10x2 dices matrix with min value of 1 and max value of 6
    dices = tf.Variable(tf.random_uniform([10, 2], minval=1, maxval=7, dtype=tf.int32))

    # Calculating the sum of rows in dices matrix
    dices_sum = tf.reduce_sum(dices, 1)
    
    # Reshaping the sum matrix to 10x1 from 1x10 concat with dices matrix
    dices_sum = tf.reshape(dices_sum, [10, 1])
    
    # Concat two matrices
    dices = tf.concat([dices, dices_sum], 1)
    
    with tf.Session() as sess:
        initialization = tf.global_variables_initializer()
        sess.run(initialization)
        print(dices.eval())









    



[[4 3 7]
 [2 3 5]
 [4 5 9]
 [3 3 6]
 [2 3 5]
 [1 6 7]
 [2 4 6]
 [1 6 7]
 [2 5 7]
 [6 2 8]]